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ABSTRACT 

The heightened interest in college program review is 
traced to a widespread interest in improving educational quality and 
the need to respond creatively to financial constraints and external 
expectations for accoiutability. Current program reviews have also 
been designed to aid in decision making about resource reallocation 
and program discontinuance. Most academic program reviews draw on one 
or more of several formal evaluation models: goal-based, responsive, 
decision-making, or connoisseurship models. The underlying objective 
of quality is defined by four different perspectives: the 
reputational view, the resources view, the outcomes view, and the 
value-added view. Most institutions assess quality by adopting 
aspects of all four views. The continued existence and growth of 
program review processes suggest that the efforts are supported and 
that the results can be beneficial, but more systematic study of 
their effects is needed. (LB) 
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ithin the last few years, the 
role of academic program review has 
emer^ged as one of the most salient issues 
in American higher education. Nestled 
within a context of accountability, pro- 
gram reviews have become a dominant 
and controversial activity at the institu- 
tional authority varies greatly, higher edu- 
cation agendes in all 50 states now con- 
duct state-level reviews; 28 of those agen- 
des have authority to discontinue pro- 
grams. Moreover, a m^ority of the 
inulticampus systems have introduced 
program reviews, and over three-fourths 
of the nation*s collies and universities 
employ some type of program review 
(Barak 1982). The heightened interest in 
program review can be traced to a wide- 
spread interest in improving program qual- 
ity and the need to respond creatively to 
severe financial constraints and to exter- 
nal constituendes* expectations for 
accountability. 

The fiterature contains a generous 
amount of controversy regaurding the pur- 
poses, processes, and outcomes of pro- 
gram review. The intent of this monogi^h 
is to illununate this terrain: to capture the 
diverse institutional approaches to review, 
to examine the central issues, and to 
reflect on ways in which program review 
might be improved. Toward that end, the 
report is based on a review of the fitera- 
ture and an analysis of program review 
practices at 30 representative institutions. 



l^t Disdiigiiishes Current 
Rnogram Reviews? 

Q>lleges and umVersities have a long- 
standing tradition of program evaluation, a 
tradition that can be traced from colom'al 
and antebellum colleges to modem Ameri- 



can universities. Until well into this cen- 
tury, program reviews were viewed lar^ly 
as internal matters, initiated most often to 
reform and revitalize the curriculum. The 
idea that program reviews should be con- 
ducted to demonstrate accountability to 
external constituencies is a phenomenon 
of the twentieth century. The gradual 
development of regional and professional 
accrediting associations and the creation 
of statewide governing and coordinating 
boards are at least partly the result of a 
belief that programs must be responsive to 
the needs and expectations of external as 
well as internal audiences. 

Espedally in the last several years, pro- 
gram reviews- have been designed to 
achieve another major objective: aiding 
those making dedsions about the realloca- 
tion of resources and program discontinu- 
ance. Thus, a broad range of expectations 
now exists for program review in higher 
education. Program improvement, 
accountabifity to external constituencies, 
and resource reallocation are the purposes 
dted most often. Despite this growth in 
expectations, fittle evidence suggests that 
an evaluation system can be designed to 
address multiple purposes simultaneously. 
It is espedally difficult to pursue both pro- 
gram improvement and resource realloca- 
tion at the same time (Barak 1982; George 
1982), and an institution's interests are 
served best if reviews focused on program 
improvement are conducted separately 
from those concerned with reallocating 
resources. 



What Do Formal Evaluation 
Models Contribute? 

Program reviews at most institutions draw 
heavily on one or more of several models: 
goal-based, responsive, decision-making, 
or connoisseurship. Although these mod- 
els are seldom explicitly identified in 



descriptions of institutional review pro- 
cesses, they can be inferred from the pro- 
cedures used. 

The goal-based mode! has had the most 
influence, offering the advantages of sys- 
tematic attention to how a program has 
performed in relation to what was 
intended and of a concern for the factors 
contributing to success or failure. The 
characteristic of the responsive model that 
has influenced program reviews in higher 
education is the attention given to pro- 
gram activities and effects, regardless of 
what its goals might be. The central con- 
cern of an evaluation, according to a pro- 
ponent of responsive evaluation, ought to 
be the issues and concerns of those who 
have an mterest in the program, not how a 
program has performed relative to its for- 
mal goal statements. 

The major contribution of the decision- 
making model to program review in higher 
education is the explidt attempt to link 
evaluations with decision making, thus 
focusing the evaluation and increasing the 
Ukelihood that results will be used. The 
connoisseurship model of evaluation has a 
long tradition in higher education. It relies 
heavily on the perspectives and judgments 
of experts, which are valued because of 
the individual's assumed superior knowl- 
edge and expertise and a commonly 
shared value system (Gardner 1977). 




How Should Quality Be 
Assesse d ? 

The assessment of quality has generated 
more confusion and debate than any other 
issue for those engaged in program review. 
Pressure to define what quality means and 
what types of information should be col- 
lected has always existed, but interest has 
been heightened by the relatively recent 
emphasis on program review for resource 
reallocation and retrenchment. 

R)ur different perspectives have been 
offered on how quality should be defined: 
the rtputational view, the resources view, 
the outcomes view, and the value-added 
view. The reputational view assumes that 
quality cannot be measured directly and is 
best inferred through the judgments of 
experts in the field. The resources view 
emphasizes the human, financial, and 
physical assets available to a program. It 
assumes that high quaUty exists when 
resources like excellent students, produc- 
tive and highly qualified faculty, and mod- 
em facilities and equipment are prevalent. 

The outcomes view of quaUty draws 
attention from resources to the quality of 
the product. Faculty publications, stu- 
dents* accomplishments following gradua* 
tion, and employers* satisfaction with pro- 
gram graduates, for example, are indica- 
tors used. The problem with the outcomes 
view is that the program*s contribution to 
the success of graduates, for example, is 
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not isolated. It is assumed that if the gradu- 
ate is a success, the program is a success. 

The value-added view directs anention 
to what the institution has contributed to a 
student*s education (Astin 1980). The 
focus of the value-added view is on what a 
student has learned while enrolled. In 
turn, programs are judged on how much 
they add to a student *s knowledge and 
personal development. The difficulty with 
this view of quah'ty is how to isolate that 
contribution. 

Most institutions assess quality by 
adopting aspects of all four views. The 
assumption is that quality has multiple 
dimensions and, in turn, that multiple indi- 
cators should be used for its assessment. 
A large number of quantitative and quali- 
tative indicators have been suggested for 
making such assessments (Clark, Hart- 
nett, and Baird 1976; Conrad and Black- 
bum 1985). 



Do Program Reviews Make a 
Difference? 

Perhaps the most significant issue relating 
to program review is the effect of the con- 
siderable activity at all levels of higher 
education. The assessment of impact 
requires that attention be given to the 
longer-term effects of decisions that are 
made, that is, whether a program is 
stronger, more efficient, or of higher qual- 
ity. The major criterion to use in assessing 
impact is whether an evaluation makes a 
system function better (Cronbach 1977). 

Only a few studies have analyzed 
impact systematicany. The University of 
Caliform'a (Smith 1979) and the University 
of Iowa (Barak 1982) benefited from pro- 
gram reviews, including providing a stimu- 
lus for change and improving knowledge 
among decision makers about programs. 
Not all analyses of impact are as positive, 
however. A small number of studies (Sku- 
bal 1979; Smith 1979) have focused on 
cost savings and have found that little 
money is saved— that, in fact, reviews fre- 
quently require an increased commitment. 
Program reviews can have negative 
effects— unwarranted anxiety, diversion of 
time from teaching and research, and 
unfulfilled promises and expectations 
(Seeley 1981). 

The continued existence and growth of 
program review processes suggest that 
such efforts are supported and that the 
results can be beneficial. Given the pleth- 
ora of program reviews at all levels of 
higher education, the need to study the 
effects of such reviews more systemati- 
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